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ABSTRACT 

In a study to evaluate instructional material as a 
function of teacher-manifai' format and material complexity, 52 
preservice special education teacjiers, 19 to ^4 years of age, 
positively rated simple material when, it was accompanied by an 
illustrated poorly organized manual. Inclusion of a well organized, 
illustrated manual elinjinated differences as a fiinction of 
complexity. .The findings may reflect inexperience of tjae subject 
population in evaluating materials. .Experienced teachers might be 
expected to make more critical judgments according to denjeuids of the 
teaching situation. . (For related materials, please see EC 052 343)|. - . 
(Author/MC) 
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Abstract 

Evaluation of instructional material was studied as a func- 
tion of teacher manual format and material complexity. Results 
from assessments by 52 teachers-in-training indicated more posi- 
tive ratings of simple than complex material. This influence 
was only evident, however, wh>n the material was accompanied by 
an unillustrated, poorly organized manual. Inclusion of an il- 
lustrated, well organized manual eliminated differences as a func- 
tion of complexity. No main effects differences were evident 
between manual formats. 
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One of the persisting problems in Special Education has been the 
evaluation of instructional materials. Various models and approaches 
have been articulated which may promote a more organized and effective 
effort focusing on this task (e.g. Moss, 1968; Mclntyre & Nelson, 1969; 
Drew & Martinson, 1971). Systematic approaches, however, remain pri- 
marily unimplemented on a broad scale basis. As a consequence, a major 
portion of instructional materials are being adopted or eliminated pri- 
marily on the basis of producers' statements as to what they will or 
will not accomplish. 

Knowledge concerning factors which may influence evaluative judge- 
ment is a critical precedent to adoption of an evaluation model. It 
seems unlikely that feasibility will permit evaluation of a large number 
of items by a central agency. This leaves primarily the teacher - consumer, 
on an individual basis, to assess the worth of materials which are pre- 
sented for review. Even if a model were implemented, which as a common 
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frame of reference would serve to reduce evaluation variability, this 
may account for only a small portion of the total variance. Other fac- 
tors relative to the design characteristics of materials may be generating 
substantial variance which Is unaccounted for. Evaluation In the absence 
of knowledge concerning the influence of these factors may be so un- 
stable as to be rendered meaningless* This would seem to highlight 
the need for systematic investigation of generic characteristics 
of instructional material design which may influence evaluative 
assessment* The purpose of the present study was to explore the 
effects of two such design characteristics, teacher manual format 
and material complexity, on the evaluation of instructional material- 
Method 

Subjects 

Subjects for this study were 32 undergraduate special education 
teachers- in* training. All were majors in Special Education who were 
currently enrolled in, or had just completed the introductory teacher-* 
training course sequence at the University of Texas at Austin* The 
mean CA for the sample was 20*8 years with a range from 19 to 24 years. 
Materials 

Two pieces of instructional material were selected from the col- 
lection at the Special Education Instructional Materials Center at the 
University of Texas (UTSEIMC)* One item was chosen as a complex material 
based on judgements by two curriculum specialists that its intended 
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objectives and procedures demanded a thorough examination prior to use* 
Similarly, the second material was selected as less complex on the basis 
of its intended objectives and procedures being judged obvious from 
cursory examination. 
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The commercially prepared teacher's manuals accompanying these 
materials were redesigned resulting In two versions of each. Experi- 
mental Form A of each manual was 11 lustra ted , typed double-spaced » em- 
ployed Indented paragraphs and clearly differentiated sections, and 
accentuated section headings with upper-case type, approximating bold 
face print. Experimental Form B was not Illustrated, was typed single- 
spaced, did not make use of indented paragraphs or discrete sections, 
and was uniformly printed in lower-case type. The substantive content 
of the original teacher manuals was maintained in both Form A and B 
of the experimental manuals, thus holding constant the actual informa- 
tion available to subjects. 

The evaluation instrument required subject's response to ten items 
relating to the material and the teacher's manual. Items were complete 
sentences designed specifically to explore the contribution of the teacher' 
manual to the perceived utility of the material. Subjects indicated the 
degree to which they were in agreement vrith each statement using a 1 to 6 
scale (1 » low, 6 « high). All items were worded such that a rating 
indicating a high degree of agreement represented a positive assessment 
of the material and a low rating suggested a negative assessment. 
Procedure 

TWenty-six subjects were randomly assigned to a group that received 
Form A of the teacher's manual. The remaining 26 subjects received Form 
B of the manual. Subjects in both groups examined both the simple and com-* 
plex material. Sequence of presentation was randomly counterbalanced to 
control for possible order effects. 
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Subjects were individually brought to a minimal distraction room 
(Drew, 1969) where they were Informed that the experimenter was conduct- 
ing research on Instructional materials. The first of the two materials 
was then presented with Instructions to the subjects that they were to 
examine It for a five minute period while the experimenter was absent. 
When this Interval had lapsed, the experimenter entered the room and 
removed the material. At that time the subject was presented the evalua- 
tion form and asked to complete it during a second five minute period. 
When the subject had completed this evaluation the second material was 
presented and the same procedure repeated. 

Results and Dl. cusslon 

The criterion measure for each subject was a total score obtained 
by summing each item rating across all ten items. This produced a score 
that indicated positive assessment as it approached 60 and less positive 
assessment as it approached 10. Table 1 summarizes mean scores by conditions. 
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Data comparing performance as a function of teacher manual format 
were analyzed using a Mann-Whitney U test for large samples (Slegel» 1956). 
Results of this analysis indicated no significant differences in evaluation 
ratings as a function of teacher manual format. 

Repeated measures comparisons as a function of material complexity 
were made using the Wllcoxon matched pairs signed ranks test (Slegel, 1956). 
Results of this analysis indicated significantly more positive evaluations* 
of simple than complex material when Form B of the teacher manual was used 
(P < .02). Evaluations did not, however, differ significantly as a function 
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of material complexity when the Form A manual was used. 

The absence of significant differences as a function of teacher 
manual format Is of particular Interest. Evaluative ratings by these 
subjects appear to be somewhat Insensitive to rather radical manual 
organizational differences. The present data do not, however. Indi- 
cate that manual format was totally without Influence. Subject material 
assessment as a function of material complexity seemed to be related 
to manual organization. Subjects receiving Form B of the manual 
(unlllustrated and "poorly organized") rated the simple material more 
positively than the complex material. Subjects receiving Form A of 
the manual (Illustrated and "well organized") did not differentially 
rate the simple and complex materials. This result Indicates that 
when objectives were not readily evident from the manual (Form B), a 
more positive rating was given the material that permitted purpose 
Identification from cursory examination. These findings. In fact, 
are consistent with Intuitive expectations In that subjects apparently 
utilized the well-organized, more attractive manual Independent of 
actual material characteristics. However, availability of only a sub- 
standard teacher's manual served to focus attention towards the material, 
eliciting more discerning material evaluation. 

These findings may to some degree reflect the relative Inexperience 
In materials evaluation characterizing the subject population partici- 
pating In this study. Experienced teachers might be expected to make 
more critical judgements of relevant material attributes In that the 
reality of the teaching situation demands Incisive materials appraisal. 
Consequently future Investigation assessing the effects of variation In 
teaching experience on material evaluation would appear worthwhile In the 
study of factors Influencing materials selection. 
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Table 1 



Mean Evaluation Scores by Teacher Manual 
Format and Material Coraplexity 





Manual 


Format 


Material 


Form A 


Form B 


Complexity 




Simple 


46.12 


48.54 


Complex 


45.35 


44.15 
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